yt 1
Knowing When to Quit: A Principled Framework for Dynamic Abstention in LLM Reasoning
Davidov, Hen, Cohen, Nachshon, Kalinsky, Oren, Fairstein, Yaron, Kushilevitz, Guy, Yazdi, Ram, Rebeschini, Patrick
Large language models (LLMs) using chain-of-thought reasoning often waste substantial compute by producing long, incorrect responses. Abstention can mitigate this by withholding outputs unlikely to be correct. While most abstention methods decide to withhold outputs before or after generation, dynamic mid-generation abstention considers early termination of unpromising reasoning traces at each token position. Prior work has explored empirical variants of this idea, but principled guidance for the abstention rule remains lacking. We present a formal analysis of dynamic abstention for LLMs, modeling abstention as an explicit action within a regularized reinforcement learning framework. An abstention reward parameter controls the trade-off between compute and information. We show that abstaining when the value function falls below this reward strictly outperforms natural baselines under general conditions. We further derive a principled and efficient method to approximate the value function. Empirical results on mathematical reasoning and toxicity avoidance tasks support our theory and demonstrate improved selective accuracy over existing methods.
- Europe > Monaco (0.04)
- Asia > Middle East > Jordan (0.04)
- North America > Canada > Quebec > Montreal (0.04)
- Europe > Switzerland (0.04)
- North America > United States (0.04)
- Asia > China > Beijing > Beijing (0.04)
72dad95a24fae750f8ab1cb3dab5e58d-Supplemental-Conference.pdf
Forclassification tasks, asmentioned inboth ofthe experiments, we could easily adopt a human-machine collaboration framework: since our model is capable of conveying the prediction confidence along with the prediction itself, we could pass thecases where themodel islessassertivetohumans forfurther evaluation. Thistraitisespecially valuable for classification tasks with exceptionally imbalanced data,e.g., fraud detection, and ad click-through rate prediction, where the volume of one class could be orders of magnitude more than the other.
72dad95a24fae750f8ab1cb3dab5e58d-Paper-Conference.pdf
These additive-noise models areprimarily focusing onaccurately estimating theconditional mean E[y|x], while paying less attention to whether the noise distribution can accurately capture the uncertainty ofy given x. For this reason, they may not work well if the distribution ofy given x clearly deviates from the additive-noise assumption. For example, ifp(y|x) is multi-modal, which commonly happens when there are missing categorical covariates inx, then E[y|x] may not be close to any possible true values ofy given that specificx.
- North America > United States > Virginia > Fairfax County > Fairfax (0.04)
- North America > United States > Massachusetts > Middlesex County > Cambridge (0.04)
- Asia > China > Beijing > Beijing (0.04)
- Asia > South Korea > Seoul > Seoul (0.04)
- Asia > China > Guangxi Province > Nanning (0.04)
- Europe > Switzerland > Zürich > Zürich (0.14)
- North America > United States > Pennsylvania > Philadelphia County > Philadelphia (0.04)
- North America > United States > Illinois > Cook County > Chicago (0.04)
- (6 more...)
Prediction and Change Detection
We measure the ability of human observers to predict the next datum in a sequence that is generated by a simple statistical process undergoing change at random points in time. Accurate performance in this task requires the identification of changepoints. We assess individual differences between observers both empirically, and using two kinds of models: a Bayesian approach for change detection and a family of cognitively plausible fast and frugal models. Some individuals detect too many changes and hence perform sub-optimally due to excess variability. Other individuals do not detect enough changes, and perform sub-optimally because they fail to notice short-term temporal trends.